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Abstract 

This  paper  reviews  interconnection  networks  and  describes  packaging  of  the 
NYU  Ultracomputer.  The  packaging  is  based  on  the  'button'  technology  developed  by 
TRW  and  adopted  by  MCC  for  its  ES-Kit  modules.  This  paper  presents  the  basic  ES-Kit 
modules  needed  to  assemble  several  sizes  of  Ultracomputers.  It  also  reviews 
Ultracomputer  prototypes  and  previews  future  prototypes. 
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1.    Introduction 


The  Ultracomputer  is  a  shared  memory  multiprocessor  computer  (Figure  1.1). 
Since  the  beginning  of  the  NYU  Ultracomputer  project  in  1979,  packaging  of  the 
interconnection  network  has  been  the  fundamental  limitation  to  the  maximum  realizable 
size  and  performance  of  an  Ultracomputer.  In  "Wireability  of  an  Ultracomputer,"  the 
wiring  of  a  4096  processor  (PE)  Ultracomputer  was  described  (Figures  1.2  and  1.3).  It 
was  shown  how  a  three  dimensional  wiring  scheme  for  an  omega  interconnection 


network  could  produce  short  interstage  wires  (Figures  1.4  and  1.5).  In  "An  Overview  of 
the  NYU  Ultracomputer  Project,"^  the  computational  speeds  and  challenges  of 
parallelism  were  addressed.  In  this  paper  a  simple  modular  packaging  scheme  for  an 
Ultracomputer  based  on  the  'button'  technology  ^'^  developed  by  TRW  for  the  VHSIC 
Phase  2  Packaging  Project  will  be  presented.  MCC,  the  Microelectronics  and  Computer 
Technology  Corporation,  adopted  this  technology  for  its  Experimental  Systems  (ES-Kit) 
modules  ^•^.  The  ES-Kit  research  project's  goal  is  to  develop  rapid  implementation  of 
low  cost  prototyping  of  parallel  computing  systems.  The  packaging  scheme  described 
here  bases  its  design  on  a  simple  set  of  Ultracomputer  ES-Kit  modules.  Existing  NYU 
Ultracomputer  prototypes  are  also  presented  in  this  paper  along  with  a  preview  of 
proposed  prototypes. 
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16X16  Omega  Network  (2   Dimension) 
Using   2X2  Switches 
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64X64  Omega  Network  (3  Dimension) 
Using  2X2  Switches  on  8X8  boards 
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2.     Interconnection  Networks 

The  Ultracomputer,  a  high  performance  multiprocessor  shared  memory 
architecture,  requires  an  interconnection  network  to  connect  the  processors  (PEs)  to  the 
memories  (MMs).  The  choice  of  a  network  affects  the  performance  and  size  of  an 
Ultracomputer.  As  used  in  Ultra  II,  described  later,  a  Bus  (Figure  2.1)  is  one  possibility 
for  such  a  network.  Its  advantage  is  that  the  number  of  wires  needed  is  proportional  to 
N,  the  number  of  PEs.  Its  limitations  are  the  fixed  bus  bandwidth  and  resulting  loading. 
Due  to  these  limitations  the  number  of  processors  supported  by  a  bus  is  in  the  order  of 
tens  of  PEs.  A  Cross  Bar  switch  with  fanout  and  multiplexing  capability  at  each  module 
(Figure  2.2)  is  another  possible  network.  It  has  excellent  connectivity  and  bandwidth, 
but  requires  order  N^  wires.  The  maximum  number  of  processors  buildable  with  such  a 
network  is  about  a  hundred. 
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For  thousands  of  processors  an  omega  interconnection  network  is  a  choice  to  be 
considered  (Figure  2.3).  This  network  has  bandwidth  proportional  to  N,  wiring  order  N 
log  N,  and  an  access  latency,  ignoring  queueing  delays,  proportional  to  log  N  .  Network 
combining  of  requests  and  processor  caches  help  to  minimize  the  limitation  of  access 
latency.  The  network  switch  is  a  critical  element  of  this  network  architecture.  It  must 
route,  queue  and  combine  requests.  Fetch  and  Add,  which  is  a  combinable  request  and 
a  required  operation  of  the  Ultracomputer,  must  also  be  supported  by  the  switch.  This 
requires  the  switch  to  support  arithmetic  operations.  Also,  to  minimize  routing  delays 
and  permit  one  clock  latency  through  the  switches  with  combining,  packets  and  pinouts 
must  be  wide  enough  to  provide  full  address  and  request  information  simultaneously. 
This  requirement  means  that  for  a  2X2  switch  the  number  of  pins  needed  must  be  four 
times  about  40  signals  for  each  direction  including  flow  control,  or  about  320  signal  pins 
per  switch  chip.  A  package  must  provide  about  400  pins  to  contain  one  2X2  switch 
(Figure  2.4).  At  NYU  a  VLSI  effort  has  been  underway  for  the  design  of  this  2X2 
switch^'®.  A  modified  Ultra  II  prototype  is  presently  in  operation  with  an  NYU  non- 
combining  VLSI  2X2  switch. 
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Following  are  a  group  of  Figures  which  demonstrate  how  a  large  omega  network 
can  be  packaged.  Depicted  in  Figure  2.5  are  several  topologically  equivalent  16X16 
omega  networks.  The  interstage  wiring  patterns  vary  but  the  only  difference  is  in  the 
placement  of  the  2X2  switches.  Figures  2.6  and  2.7  show  how  the  wiring  patterns  vary 
for  a  64X64  omega  network  depending  on  the  grouping  of  2X2  switches.  For  a  VLSI 
implementation  the  tiling  of  4X4's  to  make  a  64X64  should  produce  a  smaller  geometry 
(Figure  2.6).  In  Figure  2.8  the  three  dimensional  package  of  these  64X64  networks  is 
shown.  Figure  2.9  demonstrates  how  a  stack  of  4X4  switches,  with  inputs  on  top  and 
outputs  on  bottom,  can  be  configured  for  a  64X64  omega  network.  This  scheme 
requires  routing  channels  between  the  three  layers  and  will  be  described  in  detail  in  the 
next  section. 


16  X  16  Omega  Interconnection  Networks 


Shuffle 


Binary 

Figure  2.5 


4  X4's 


11 


64  X  64  Omega  Interconnection  Network 
using  4  X  4's 


Figure  2.6 
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64  X  64  Omega  Interconnection  Network 
using  8  X  8's 


Figure  2.7 
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3  Dimension  64  X  64  Switches 
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3  Dimension  64  X  64  Switch  Stack 


Figure  2.9 
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3.     Ultracomputer  'Button'  Technology  Modules 

A  simple  modular  packaging  scheme  for  an  Ultracomputer  using  the  'button' 
technology  developed  by  TRW  will  now  be  presented.  Note:  a  recent  AT&T  Bell 
Laboratories  developed  Elastomeric  Conductive  Polymer  Interconnect  (ECPI)^  exhibits 
an  equivalent  three  dimensional  continuity  property  and  could  serve  as  a  replacement 
for  the  'buttons'  in  what  follows.  The  ECPI  is  a  thin  sheet  of  resilient  silicone  rubber 
insulation  imbedded  with  densely  populated  conductors. 

The  'button'  technology  has  been  designed  to  facilitate  the  interconnection  of 
stacked  modules  with  separate  sets  of  connections  on  the  top  and  bottom  faces  of  each 
module.  If  the  'button'  technology  ES-Kit  packaging  system,  developed  by  MCC 
corporation,  is  used,  a  simple  set  of  'Ultra'  modules  can  be  devised  for  the  assembly  of 
several  sizes  of  Ultracomputers. 

Three  basic  module  types  are  necessary  to  form  a  four  processor  Ultracomputer, 
a  processor  (PE)  module,  a  memory  (MM)  module,  and  a  four  input-four  output  switch 
(SW)  module.  The  basic  ES-kit  module  contains  four  groups  of  106  connection  pads  on 
both  top  and  bottom  faces  of  the  module,  two  groups  on  each  end  of  the  7  inch  side  of 
the  7  inch  by  5.5  inch  module.  With  discrete  components  an  Ultracomputer  module  will 
require  a  double  (7  inch  by  10  inch)  size  module  (see  Appendix  for  detailed  drawings  of 
board  and  'buttons').  For  clarity  in  this  description,  it  is  assumed  that  coplaner  modules 
are  centered  on  a  square  grid.  The  center-to-center  distance  serves  as  a  unit  of  length 
in  what  follows. 
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The  wiring  of  both  the  PE  and  MM  modules  is  configured  so  that  their  external 
connections  are  located  on  one  corner  of  the  top  face  and  also  wired  to  a  corner  of  the 
bottom  face  rotated  by  90  degrees.  The  remaining  three  top  corner  connections  are 
separately  wired  from  a  top  corner  to  a  90  degree  rotated  bottom  corner  (Figure  3.1). 
With  this  wiring  scheme  all  external  connections  of  four  stacked  PEs  or  MMs  are  brought 
to  both  the  top  and  bottom  faces  of  the  stack.  A  stack  of  up  to  four  PEs  and  MMs  forms  a 
useful  assembly  for  diagnosis  of  PE  /  MM  operation.  A  one  PE  /  MM  combination  is 
shown  in  Figure  3.4. 


Top  Side 


Bottom  Side 
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Memory  Module     (MM) 


Figure  3.1 

The  four  input-four  output  switch  (SW)  module  has  its  inputs  wired  to  each  of  the 
four  top  corners  and  its  outputs  to  the  corresponding  four  bottom  corners  (Figure  3.2). 
A  stack  of  nine  modules,  (4PEs,  1SW,  and  4MMs)  comprises  a  4PE  Ultracomputer 
(Figure  3.3  and  Figure  3.4).  This  combination  is  expanded  in  Figure  3.4  to  show  the 
'button'  technology  used  to  connect  adjacent  boards. 
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4.     16  to  256  PE  Ultracomputers 

Incorporating  simple  wire  routers  with  tine  three  module  types  described  above 
permits  still  larger  Ultracomputers  to  be  assembled.  Those  easily  built  are  successive 
multiples  of  four,  i.e.  4,  16,  64,  256,  etc.  The  most  straightforward  packaging  for  larger 
Ultras  require  an  additional  component,  routers  that  connect  4X4  switch  stages.  Two 
types  of  routers  are  added  between  stages,  one  vertical  and  the  other  horizontal.  These 
components  have  a  small  dimension  of  one  unit  and  a  large  dimension  of  two,  four, 
eight,  etc.,  units  depending  on  the  number  of  stages  required  for  a  particular  size 
Ultracomputer. 

For  example,  to  build  a  16PE  Ultracomputer,  only  two  kinds  of  routers  are 
needed,  a  horizontal  router  (hR),  one  unit  high  by  two  units  wide,  and  a  vertical  router 
(vR)  two  units  high  by  one  unit  wide  (Figure  4.1).  Two  of  each  are  required.  The 
function  of  these  routers  is  simply  to  distribute  each  of  the  four  outputs  of  a  4X4  switch 
stage  to  one  input  of  each  of  four  different  4X4  switches  of  the  next  stage. 


Horizontal  Router    (hR) 


Vertical  Router    (vR) 


Figure  4.1 
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One  composition  of  a  16PE  Ultracomputer  would  have  four  stacks  each  with 
4PEs  and  one  SW,  on  a  layer  of  two  hRs,  followed  by  a  layer  of  two  vRs,  and  finally  four 
stacks  each  having  one  SW  and  4MMs  (Figures  4.2  and  4.3). 

16  PE  Ultracomputer  (with  hR  and  vR) 
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16  PE  Ultracomputer  (with  hR  and  vR) 
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It  is  also  possible  to  build  a  16PE  Ultra  with  only  one  long  router  between  the 
stages,  either  a  horizontal  router  (HR)  one  unit  high  by  four  units  long,  or  a  vertical  router 
(VR)  four  units  high  by  one  unit  long.  This  is  advantageous  for  16PE  and  larger 
Ultracomputers,  since  it  reduces  the  number  of  routers,  and  thickness  of  the  stack.  For 
the  following  16PE  implementation  only  one  layer  of  one  HR  router  is  required  (Figures 
4.4,  4.5  and  4.6).  Such  a  router  distributes  the  four  outputs  of  a  4X4  switch  to  one  input 
each  of  four  4X4  switches  of  the  next  stage.  Figure  4.6  shows  how  such  an  assembly  is 
physically  held  together  by  nuts  and  bolts.  The  nuts  and  bolts  also  facilitate  the 
electrical  connection  between  'buttons'  and  modules. 

16  PE  Ultracomputer  (with  HR) 


Horizontal   Router 
HR 


■^^ja^jg-^sa^ 


MMs 
SWs    (4X4) 
PES 


Figure  4.4 


21 


16  PE  Ultracomputer  (with  HR) 


Figure  4.5 
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16  PE  Ultracomputer  (with  HR) 
Showing  Buttons,  Nuts  and  Bolts 
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Figure  4.6 
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In  a  64PE  Ultracomputer  (Figures  4.7,  4.8  and  4.9),  both  the  HR  and  VR  routers 
are  needed,  requiring  a  stack  thickness  of  13  (4PEs,  SW,  VR,  SW,  HR,  SW  and  4MMs). 
Figure  4.8  shows  the  detail  wiring  pattern  of  the  HR  or  VR  and  the  proposed  Ultra  III 
packaging.   Figure  4.9  depicts  quantities  and  details  of  the  components. 
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64  PE    Ultra  III  Package 
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64  PE  Ultracomputer  (Components) 
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Figure  4.9 
A  256PE  Ultracomputer  will  include  another  stage  of  4X4  SWs  and  two  larger 
wire  routing  modules.  A  large  vertical  router  (LVR),  eight  units  high  by  one  unit  wide, 
and  a  large  horizontal  router  (LHR),  one  unit  high  by  eight  units  wide.  Four  64PE 
Uitracomputers  are  grouped  in  a  two  by  two  array.  The  last  4X4  SW  stage  and  4MMs 
are  moved  back  and  a  stage  of  4X4  SWs,  a  large  vertical  router,  and  a  large  horizontal 
router  are  inserted  into  the  stack.   Its  composition  is  4PEs,  SW,  VR,  SW,  HR,  SW,  LVR, 
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LHR,  SW  and  4MMs,  or  16  layers  in  a  stack  (Figure  4.10).  An  alternative  stacking  witli 
the  same  modules  is  4PEs,  SW.  VR.  SW,  LVR,  LHR,  SW,  HR,  SW  and  4MMs  (Figure 
4.11). 

256  PE  Ultracomputer 


j^^^^^^^ 


'^ — n 


MMs 


LHR    or    LVR 

fo" 

o 

o 

o 

o 

o 

O       O       lO       O 

fi 

~~s 

o 

o 

o 

o 

o 

o| 

u 

o 

0     01     o     o 

1 

Swap 

Figure  4.10 


27 


256  PE  Ultracomputer 
Alternative   Stacking 
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5.     Larger  Ultracomputers 

Larger  Ultracomputers  are  possible  with  'button'  packaging.  It  should  be  noted 
that  the  longest  wire  in  simple  routers  is  half  the  length  of  the  router  and  also  that  large 
Ultracomputers  do  not  require  one  piece  long  routing  boards.  Instead,  these  boards  can 
be  made  of  layers  of  smaller  boards  and  'buttons'.  When  half-size  boards  are  used  to 
replace  a  full-size  board,  two  additional  'button'  layers  and  three  layers  of  overlapped 
half-size  boards  filled  with  either  longer  'button'  connectors  or  quarter-size  null  boards 
are  required  (Figure  5.1).  Similarly,  these  half-size  boards  can  in  turn  be  made  of 
quarter-size  boards.  This  process  ends  with  the  hR  or  vR.  Such  substitutions,  aside 
from  eliminating  the  requirement  for  large  boards,  reduce  the  density  of  wires  at  the 
center  of  the  remaining  boards,  but  at  the  cost  of  more  layers  of  'buttons'  and  boards. 

Large  Routers  Assembled  using  Small  Routers 
LHR  or  LVR 


hR  or  vR 


Notes:    Drawing  Is  a  cross  section.    On  these  routing  boards, 
half  the  connections  are  straight-through  and  are  not  shown. 

Figure  5.1 
A  1024  PE  and  a  4096  PE  configurations  with  the  above  modules  and  routers  will 
produce  physically  large,  relatively  thin,  Ultracomputers  (Figures  5.2  and  5.3).  Smaller 
'button'  boards  with  more  connections  and  higher  densities  coupled  with  more  PEs, 
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MMs  and  larger  omega  networks  per  SW  (utilizing  4X4,  8X8  or  larger  switch  chips)  will 
produce  physically  smaller  Ultracomputers.  For  example,  if  the  number  of  connections 
per  'button'  board  doubles,  then  an  8X8  omega  network  could  be  packaged  on  a  SW 
module.  The  larger  omega  network  per  SW  module  will  then  reduce  the  number  of  inter 
SW  wire  router  boards  required  for  a  given  size  Ultracomputer. 

The  complexity  of  packaging  and  wiring  omega  interconnection  networks  will 
always  limit  the  size  of  an  Ultracomputer,  however,  it  will  still  be  possible  to  build  them 
with  thousands  of  processors. 

1024  PE  Ultracomputer 


n-nn-nn— nrr-nn-nn-nn-nir-nn-nn-nii-nir-iin— nir 


UUUUUUL 

iiyiyuuuLiiy 

ODQDDDE 

9DDDQQDQ 

DOODDDE 

IQDDQDDO 

pDDDDDE 

gDDDDDDd 

DDDDDDE 

gDQQDDDO 

QDDDDQE 

idddddqQ 

QDDDDDE 

idddqddQ 

-innnnnor 

lonnnoniri 

a 


S==' 


SWs 


Figure  5.2 


30 


4096  PE  Ultracomputer 
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6.     NYU  Ultracomputer  Prototypes 

The  NYU  Ultracomputer  prototype  effort  began  in  1982  with  the  design  and 
implementation  of  a  two  processor  Ultracomputer  (Ultra  I,  Figure  6.1).  Two  Motorola 
68000  evaluation  boards  each  augmented  with  a  Motorola  68451  memory  management 
unit  were  interfaced  to  a  spare  memory  port  of  a  PUMA  (NYU  prototype  emulator  of  a 
CDC  6600)  /  DEC  PDP11  shared  memory  system.  The  PUMA  was  not  used  but  the 
PDP11  was  used  for  input  /  output.  The  purpose  of  this  prototype  and  those  that 
followed  was  to  provide  a  test  bed  for  operating  system  software  and  user  applications. 
A  highly  parallel  variant  of  the  UNIX  operating  system  was  developed  for  this  and  other 
NYU  prototypes. 


2  PE  NYU  Ultracomputer  Prototype 

Ultra  I 


68000 


1 


MMU 


68000 


MMU 


Motorola 

Evaluation 

Boards 


MNI 


8  MB 
Mem 


PP 


64  bit 


I 


Switch 


I 


PUMA 
ECL 


CDC    6600 
Emulator 


Backend    10 
Processor 


Dec    11/34 


v 

16  bit  Bus 


Decnet 


Figure  6.1 


32 


A  more  substantial  prototype  (Ultra  II,  Figure  6.2)  was  designed  and  built  during 
1983  /  84.  Several  copies  were  built.  Each  Ultra  II  contained  upto  eight  processors  (8 
PEs)  with  cache,  an  NYU  designed  Ultra  bus  and  four  two  megabyte  dynamic  ram 
memory  modules  (4  MMs  /  8  megabytes)  with  support  for  Fetch  and  Add.  The  MMs  were 
dual  ported,  the  second  port  was  again  used  by  a  PDP1 1  for  input  and  output.  The  Ultra 
bus  was  designed  with  separate  out  and  in  32  bit  data  buses  to  support  a  simple  fast 
Fetch  and  Add  operation.  The  asynchronous  Ultra  bus  also  supports  byte  write 
operations  and  contains  an  arbiter  which  arbitrates  fairly  among  the  eight  PEs,  each 
running  with  its  own  10  Mhz  clock.  The  Ultra  bus  bandwidth  is  8  megabytes  per  second. 
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The  Ultra  II  memory  modules  (MMs)  utilized  256  kb  dynamic  ram  chips  to  create 
two  1  megabyte  memory  arrays  per  MM.  The  MMs  were  dual  ported  and  contained 
individual  arbiters  so  that  the  Ultra  and  Dec  could  access  separate  MMs  simultaneously. 
A  key  to  the  memory  design  was  a  32  bit  adder  to  support  Fetch  and  Add.  The  single 
Dec  interface  module  interfaces  the  second  port  of  the  MMs  to  the  Dec  UNIBUS  and 
maps  a  12  kilobyte  space  of  the  Dec  UNIBUS's  256  kilobyte  space  onto  the  MMs 
through  12  individual  1  kilobyte  windows.  This  permits  data  transfers  directly  from  PDP 
11  Disk  to  MM. 

The  processor  design  was  the  most  complicated  since  it  had  to  incorporate  many 
features.  The  first  important  diagnostic  feature  was  a  stand  alone  processor,  with  RAM, 
ROM,  and  UART  and  capability  of  accessing  global  memory.  A  ROM  monitor  was 
provided  that  incorporated  a  Motorola  Assembler  /  Disassembler  and  an  NYU  memory 
diagnostic.  A  Motorola  68451  memory  management  unit  (MMU)  was  used  and 
supported  by  our  modified  UNIX  operating  system.  Since  the  processor  has  only  a  16 
bit  bus,  assembly  and  dissassembly  of  32  bit  data  was  provided  to  support  atomic  32  bit 
loads,  stores  and  Fetch  and  Adds.  A  32  kilobyte  2-way  set  associative,  write  through, 
least  recently  used  evict  (LRU),  virtual  address  cache  was  incorporated  to  relieve  bus 
congestion.  To  satisfy  user  floating  point  requirements,  Weitek  floating  point  chips  were 
interfaced  to  the  PE. 

Since  the  processor  selected  (Motorola  68010)  did  not  support  Fetch  and  Add 
(FAA),  the  PE  design  had  to  include  a  mechanism  for  it.  The  design  chosen  was  a  two 
register  solution.  First  the  virtual  address  of  the  FAA  operation  was  stored  in  the 
Address  Register  (AR).  Then  the  increment  was  stored  in  the  Data  Register  (DR). 
Finally,  a  load  was  issued  referencing  a  special  address.  This  caused  the  FAA  logic  to 
direct  the  stored  virtual  address  in  the  AR  through  the  MMU  to  generate  a  physical 
address,  which  would  be  placed  on  the  address  lines  of  the  Ultra  bus.  Meanwhile  the 
increment  value  in  the  DR  was  presented  on  the  data  out  lines.  The  old  value  stored  in 
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memory  was  returned  on  the  data  in  lines  and  then  to  the  processor  while  the  sum  of 
this  value  plus  the  increment  value  was  stored  back  into  the  memory. 

The  construction  of  Ultra  II  basically  consisted  of  off  the  shelf  integrated  circuit 
components  and  two  sizes  of  wire  wrapped  boards,  9  inch  by  9  inch  (LB)  and  4.5  inch 
by  9  inch  (SB).  Two  LBs  and  two  SBs  make  up  one  PE.  One  LB  and  one  SB  was  used 
for  each  2  megabyte  MM.  The  LB  of  the  MM  contained  the  memory  array,  adder  and 
data  paths.  It  was  implemented  with  printed  circuit  board  technology  after  first  being 
wirewrapped  and  tested.  The  arbiter  and  Dec  interface  boards  were  LBs.  Eight  PE 
board  sets,  four  MM  board  sets,  an  arbiter  board,  and  a  Dec  interface  board  plugged 
into  a  drawer  with  a  22  inch  by  15  inch  wirewrapped  backplane.  An  LED  display  front 
panel  giving  status  information  for  each  PE  and  a  back  panel  with  two  RS232  ports  per 
PE  completed  the  packaging  of  Ultra  II. 
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7.    Ultra  III  Prototype 

With  the  advent  of  the  megabit  dynamic  ram  and  high  performance,  true  32  bit 
RISC  processors  with  32  bit  buses,  caches,  and  support  for  multiple  outstanding 
requests  and  floating  point  coprocessors.  Ultra  II  becomes  a  less  viable  prototype  for  a 
network  Ultracomputer  and  a  new  prototype  is  needed.  The  Ultra  III  design  must  include 
a  path  toward  fabrication  of  larger  Ultracomputers.  Such  a  packaging  capability  is 
available  with  the  use  of  the  'button'  technology  and  ES-Kit  modules  as  described 
above.    A  low  risk  migration  path  toward  Ultra  III  presently  in  process  is  now  discussed. 

Three  basic  modules  must  be  designed:  a  processor  PE,  switch  SW  and  memory 
MM.  To  ease  testing  an  incremental  path  from  Ultra  II  has  been  chosen.  First  a 
synchronous  4  megabyte  memory  module  (MM)  utilizing  the  megabit  dynamic  ram  with 
a  network  protocol  and  Fetch  and  Op,  was  designed  for  Ultra  III,  and  Ultra  II  was 
retrofitted  to  accept  it.  The  retrofit  required  a  mod  to  the  arbiter  board  to  packetize  Ultra 
bus  requests.  This  produced  a  16  megabyte  Ultra  11+  (Figure  7.1),  which  is  now 
operational  and  provides  users  with  expanded  memory.  The  second  module 
implemented  was  the  Ultra  III  2  input,  2  output  switch  module  (SW).  It  utilizes  NYU 
designed  VLSI  noncombining  switch  chips  and  has  been  used  to  connect  two  Ultra  11+ 
arbiters  to  two  Ultra  III  MMs.  This  configuration  is  now  functional  as  Ultra  ll+SW 
(Figure  7.2). 
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The  first  version  of  the  Ultra  III  PE  was  built  using  the  AMD  29000  processor  and 
supporting  the  Ultra  III  network  protocol.  It  replaces  one  of  the  Ultra  ll+'s  of  the  Ultra 
ll+SW  prototype.  The  resulting  system  is  the  first  version  of  Ultra  III,  or  Ultra  IIIA  (Figure 
7.3).  It  presently  executes  stand  alone  programs;  the  operating  system  is  nearly  ready. 
The  next  version  of  the  Ultra  III  PE  is  now  in  design.  It  will  support  separate  instnjction 
and  data  caches  using  the  AMD  29062  cache  chips,  burst  mode,  multiple  outstanding 
requests,  and  Fetch  and  Add.  Control  and  network  protocol  will  be  implemented  via 
XILINX  programmable  gate  arrays. 

The  goal  of  the  next  phase  of  Ultracomputer  prototypes  is  to  develop  a  16  PE 
wirewrapped  Ultra  III  with  an  omega  interconnection  network  capable  of  combining 
(Figure  7.4)  for  the  purpose  of  testing  network  performance  and  to  facilitate  the 
fabrication  of  larger  Ultracomputers  with  larger  networks  and  substantially  more  PEs 
and  MMs.  The  16  PE  Ultra  III  will  contain  a  four  stage  network  with  32  2X2  combining 
SWs,  and  16  MMs  with  64  megabytes  of  global  memory.  The  success  of  this  design  will 
provide  specifications  for  ES-Kit  modules  from  which  a  64  PE  Ultra  III  can  be  fabricated. 
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NYU  Ultracomputer  Prototype 
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Appendix 
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Cinch  Button  Connector 
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Notes: 

Each  button  connector  has  106  signal  pins  with  controlled  impedance, 
28  ground  pins,  plus  8  power  pins.    They  also  have  two  alignment  pins  and 
three  bolt  holes. 

The  button  contacts  are  fabricated  from  2  mil  diameter  strands  of  wire 
randomly  pressed  into  a  cylinder.    Contact  is  made  between  button  and  PC 
board  'land'  by  pressure  (1-2  ounces  per  button). 

Connectors  are  made  by  pressing  buttons  into  rods  formulated  as 
spacers  between  stacked  boards.    Every  rod  has  a  button  on  both  ends. 


Figure  A2 
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